Improving reinforcement learning algorithms: Towards optimal learning rate policies

This paper shows how to use results of statistical learning theory and stochastic algorithms have a better understanding the convergence Reinforcement Learning (RL) once it is formulated as fixed point problem. can be used propose improvement RL rates. First, our analysis that classical asymptotic rate O ( 1 / N ) $O(1/\sqrt {N})$ pessimistic replaced by log β $O((\log (N)/N)^{\beta })$ with 2 ≤ $\frac{1}{2}\le \beta \le 1$ , number iterations. Second, we dynamic optimal policy for choice in RL. We decompose into two interacting levels: inner outer levels. In level, present PASS algorithm (for “PAst Sign Search”) which, based on predefined sequence rates, constructs new which error decreases faster. The proved bounds are established. an methodology selection sequence. Third, show empirically outperforms significantly standard three following applications: estimation drift, placement limit orders, execution large shares.